Field of Invention
The present invention is related to software test field, and more particular to a method of automatically generating test cases for embedded software and a device thereof. The tested program hereof is embedded software. Using this device and method can automatically generate test cases.
Description of Related Arts
With the wide application of the devices installed with embedded software, the quality of the embedded software becomes the key factor which has great influence on national economy, personal security and property security. Software testing is commonly used for improving the quality of software. But traditional methods can not thoroughly test embedded software because test cases can not cover all possible execution conditions.
The concept of symbolic execution technique was brought out around 1970. This technique has become the research hot spot recently. Dynamic symbolic execution technique is a modification of symbolic execution technique, which combines symbolic execution and concrete execution. Generally speaking, dynamic symbolic execution runs the tested software, collects path conditions while execution and generating new test cases after calculating on the path conditions. Theoretically, dynamic symbolic execution technique can test the software thoroughly because the newly generated test cases can cover all the unexecuted paths.
Due to the capability of the software and hardware of the embedded system, particularly the limitation of the computing capability of the processor and the memory space, current dynamic symbolic execution software which can run well in the general-purpose computing platform cannot be applied to the embedded software directly. Considering the complexity of the solver and the instrumentation software, directly importing the dynamic symbolic execution software from the general-purpose platform in the embedded system will cause extremely heavy developing workload.
Described in the literature SCORE: a scalable concolic testing tool for reliable embedded software, Kim and other researchers import the CREST and KLEE in the embedded system. Similar method is described in the following five papers: Concolic testing on embedded software—case studies on mobile platform programs, Industrial application of concolic testing on embedded software: Case studies, A case study on libexif by using CREST-BV and KLEE, Scalable distributed concolic testing: a case study on a flash storage platform, Concolic testing of the multi-sector read operation for flash memory file system. The aforementioned five papers describe similar methods which mainly focus on the following points: (1) Modify current dynamic symbolic execution software CREST or KLEE which runs on the general-purpose platform; (2) The source code of the tested software is instrumented using the modified dynamic symbolic execution software; (3) Run the tested software after instrumentation on the embedded system; (4) Use the modified solver to create new test cases on the embedded system. The aforementioned method has the following disadvantages: (1) The C/C++ source code of the tested software is required; (2) A lot of modifications on dynamic symbolic execution software and solver are required; (3) The symbolic execution part of the above method runs on the embedded system, which will cause huge resource consumption.
Described in the literatures Structural Testing of Executables and OSMOSE: Automatic structural testing of executables, Bardin and others bring up the idea of translating the executable code of the tested software into an intermediate language, and then symbolically executing the intermediate language in the simulator. Using this and similar method has the following disadvantages: (1) For different embedded systems, corresponding simulators need to be developed; (2) Execution conditions of the tested software in the simulator may not be the same with real execution conditions.
The literature Unleashing mayhem on binary code brings up the idea of dividing the dynamic symbolic execution into concrete execution process and symbolic execution process. Running the concrete execution process on the target system and running the symbolic execution process on any platform. This method has the following disadvantages: (1) The concrete execution part of this method includes taint tracking, dynamic binary instrumentation, virtual machine and other functional modules which consume huge resources, so it is not suitable to run on the embedded system; (2) In order to perform dynamic taint analysis on the embedded system, it is necessary for this method to import taint tracking software, dynamic binary instrumentation software, and virtual machine; (3) This method adopts hybrid symbolic execution; (4) This method designs and realizes a cross-platform lightweight RPC protocol to connect concrete execution process and symbolic execution process, but this method still is not applied to the test of embedded software.